Dopamine transients follow a striatal gradient of reward time horizons.

Animals make predictions to guide their behavior and update those predictions through experience. Transient increases in dopamine (DA) are thought to be critical signals for updating predictions. However, it is unclear how this mechanism handles a wide range of behavioral timescales-from seconds or less (for example, if singing a song) to potentially hours or more (for example, if hunting for food). Here we report that DA transients in distinct rat striatal subregions convey prediction errors based on distinct time horizons. DA dynamics systematically accelerated from ventral to dorsomedial to dorsolateral striatum, in the tempo of spontaneous fluctuations, the temporal integration of prior rewards and the discounting of future rewards. This spectrum of timescales for evaluative computations can help achieve efficient learning and adaptive motivation for a broad range of behaviors.

Comment on 'Accumbens cholinergic interneurons dynamically promote dopamine release and enable motivation'.

Acetylcholine is widely believed to modulate the release of dopamine in the striatum of mammals. Experiments in brain slices clearly show that synchronous activation of striatal cholinergic interneurons is sufficient to drive dopamine release via axo-axonal stimulation of nicotinic acetylcholine receptors. However, evidence for this mechanism in vivo has been less forthcoming. A recent paper in eLife (Mohebi et al., 2023) reported that, in awake behaving rats, optogenetic activation of striatal cholinergic interneurons with blue light readily evokes dopamine release measured with the red fluorescent sensor RdLight1. Here, we show that blue light alone alters the fluorescent properties of RdLight1 in a manner that may be misconstrued as phasic dopamine release, and that this artefactual photoactivation can account for the effects attributed to cholinergic interneurons. Our findings indicate that measurements of dopamine using the red-shifted fluorescent sensor RdLight1 should be interpreted with caution when combined with optogenetics. In light of this and other publications that did not observe large acetylcholine-evoked dopamine transients in vivo, the conditions under which such release occurs in behaving animals remain unknown.

Selective encoding of reward predictions and prediction errors by globus pallidus subpopulations.

Basal ganglia (BG) circuits help guide and invigorate actions using predictions of future rewards (values). Within the BG, the globus pallidus pars externa (GPe) may play an essential role in aggregating and distributing value information. We recorded from the GPe in unrestrained rats performing both Pavlovian and instrumental tasks to obtain rewards and distinguished neuronal subtypes by their firing properties across the wake/sleep cycle and optogenetic tagging. In both tasks, the parvalbumin-positive (PV+), faster-firing "prototypical" neurons showed strong, sustained modulation by value, unlike other subtypes, including the "arkypallidal" cells that project back to striatum. Furthermore, we discovered that a distinct minority (7%) of GP cells display slower, pacemaker-like firing and encode reward prediction errors (RPEs) almost identically to midbrain dopamine neurons. These cell-specific forms of GPe value representation help define the circuit mechanisms by which the BG contribute to motivation and reinforcement learning.

Dual credit assignment processes underlie dopamine signals in a complex spatial environment.

Animals frequently make decisions based on expectations of future reward ("values"). Values are updated by ongoing experience: places and choices that result in reward are assigned greater value. Yet, the specific algorithms used by the brain for such credit assignment remain unclear. We monitored accumbens dopamine as rats foraged for rewards in a complex, changing environment. We observed brief dopamine pulses both at reward receipt (scaling with prediction error) and at novel path opportunities. Dopamine also ramped up as rats ran toward reward ports, in proportion to the value at each location. By examining the evolution of these dopamine place-value signals, we found evidence for two distinct update processes: progressive propagation of value along taken paths, as in temporal difference learning, and inference of value throughout the maze, using internal models. Our results demonstrate that within rich, naturalistic environments dopamine conveys place values that are updated via multiple, complementary learning algorithms.

Accumbens cholinergic interneurons dynamically promote dopamine release and enable motivation.

Motivation to work for potential rewards is critically dependent on dopamine (DA) in the nucleus accumbens (NAc). DA release from NAc axons can be controlled by at least two distinct mechanisms: (1) action potentials propagating from DA cell bodies in the ventral tegmental area (VTA), and (2) activation of ß2* nicotinic receptors by local cholinergic interneurons (CINs). How CIN activity contributes to NAc DA dynamics in behaving animals is not well understood. We monitored DA release in the NAc Core of awake, unrestrained rats using the DA sensor RdLight1, while simultaneously monitoring or manipulating CIN activity at the same location. CIN stimulation rapidly evoked DA release, and in contrast to slice preparations, this DA release showed no indication of short-term depression or receptor desensitization. The sound of unexpected food delivery evoked a brief joint increase in CIN population activity and DA release, with a second joint increase as rats approached the food. In an operant task, we observed fast ramps in CIN activity during approach behaviors, either to start the trial or to collect rewards. These CIN ramps co-occurred with DA release ramps, without corresponding changes in the firing of lateral VTA DA neurons. Finally, we examined the effects of blocking CIN influence over DA release through local NAc infusion of DHßE, a selective antagonist of ß2* nicotinic receptors. DHßE dose-dependently interfered with motivated approach decisions, mimicking the effects of a DA antagonist. Our results support a key influence of CINs over motivated behavior via the local regulation of DA release.

Dual credit assignment processes underlie dopamine signals in a complex spatial environment.

Dopamine in the nucleus accumbens helps motivate behavior based on expectations of future reward ("values"). These values need to be updated by experience: after receiving reward, the choices that led to reward should be assigned greater value. There are multiple theoretical proposals for how this credit assignment could be achieved, but the specific algorithms that generate updated dopamine signals remain uncertain. We monitored accumbens dopamine as freely behaving rats foraged for rewards in a complex, changing environment. We observed brief pulses of dopamine both when rats received reward (scaling with prediction error), and when they encountered novel path opportunities. Furthermore, dopamine ramped up as rats ran towards reward ports, in proportion to the value at each location. By examining the evolution of these dopamine place-value signals, we found evidence for two distinct update processes: progressive propagation along taken paths, as in temporal-difference learning, and inference of value throughout the maze, using internal models. Our results demonstrate that within rich, naturalistic environments dopamine conveys place values that are updated via multiple, complementary learning algorithms.

Altered basal ganglia output during self-restraint.

Suppressing actions is essential for flexible behavior. Multiple neural circuits involved in behavioral inhibition converge upon a key basal ganglia output nucleus, the substantia nigra pars reticulata (SNr). To examine how changes in basal ganglia output contribute to self-restraint, we recorded SNr neurons during a proactive behavioral inhibition task. Rats responded to Go! cues with rapid leftward or rightward movements, but also prepared to cancel one of these movement directions on trials when a Stop! cue might occur. This action restraint - visible as direction-selective slowing of reaction times - altered both rates and patterns of SNr spiking. Overall firing rate was elevated before the Go! cue, and this effect was driven by a subpopulation of direction-selective SNr neurons. In neural state space, this corresponded to a shift away from the restrained movement. SNr neurons also showed more variable inter-spike intervals during proactive inhibition. This corresponded to more variable state-space trajectories, which may slow reaction times via reduced preparation to move. These findings open new perspectives on how basal ganglia dynamics contribute to movement preparation and cognitive control.

Dopamine subsystems that track internal states.

Food and water are rewarding in part because they satisfy our internal needs1,2. Dopaminergic neurons in the ventral tegmental area (VTA) are activated by gustatory rewards3-5, but how animals learn to associate these oral cues with the delayed physiological effects of ingestion is unknown. Here we show that individual dopaminergic neurons in the VTA respond to detection of nutrients or water at specific stages of ingestion. A major subset of dopaminergic neurons tracks changes in systemic hydration that occur tens of minutes after thirsty mice drink water, whereas different dopaminergic neurons respond to nutrients in the gastrointestinal tract. We show that information about fluid balance is transmitted to the VTA by a hypothalamic pathway and then re-routed to downstream circuits that track the oral, gastrointestinal and post-absorptive stages of ingestion. To investigate the function of these signals, we used a paradigm in which a fluid's oral and post-absorptive effects can be independently manipulated and temporally separated. We show that mice rapidly learn to prefer one fluid over another based solely on its rehydrating ability and that this post-ingestive learning is prevented if dopaminergic neurons in the VTA are selectively silenced after consumption. These findings reveal that the midbrain dopamine system contains subsystems that track different modalities and stages of ingestion, on timescales from seconds to tens of minutes, and that this information is used to drive learning about the consequences of ingestion.

Coupling between motor cortex and striatum increases during sleep over long-term skill learning.

The strength of cortical connectivity to the striatum influences the balance between behavioral variability and stability. Learning to consistently produce a skilled action requires plasticity in corticostriatal connectivity associated with repeated training of the action. However, it remains unknown whether such corticostriatal plasticity occurs during training itself or 'offline' during time away from training, such as sleep. Here, we monitor the corticostriatal network throughout long-term skill learning in rats and find that non-rapid-eye-movement (NREM) sleep is a relevant period for corticostriatal plasticity. We first show that the offline activation of striatal NMDA receptors is required for skill learning. We then show that corticostriatal functional connectivity increases offline, coupled to emerging consistent skilled movements, and coupled cross-area neural dynamics. We then identify NREM sleep spindles as uniquely poised to mediate corticostriatal plasticity, through interactions with slow oscillations. Our results provide evidence that sleep shapes cross-area coupling required for skill learning.

Improved inference in coupling, encoding, and decoding models and its consequence for neuroscientific interpretation.

BACKGROUND: A central goal of systems neuroscience is to understand the relationships amongst constituent units in neural populations, and their modulation by external factors, using high-dimensional and stochastic neural recordings. Parametric statistical models (e.g., coupling, encoding, and decoding models), play an instrumental role in accomplishing this goal. However, extracting conclusions from a parametric model requires that it is fit using an inference algorithm capable of selecting the correct parameters and properly estimating their values. Traditional approaches to parameter inference have been shown to suffer from failures in both selection and estimation. The recent development of algorithms that ameliorate these deficiencies raises the question of whether past work relying on such inference procedures have produced inaccurate systems neuroscience models, thereby impairing their interpretation. NEW METHOD: We used algorithms based on Union of Intersections, a statistical inference framework based on stability principles, capable of improved selection and estimation. COMPARISON: We fit functional coupling, encoding, and decoding models across a battery of neural datasets using both UoI and baseline inference procedures (e.g., ℓ1-penalized GLMs), and compared the structure of their fitted parameters. RESULTS: Across recording modality, brain region, and task, we found that UoI inferred models with increased sparsity, improved stability, and qualitatively different parameter distributions, while maintaining predictive performance. We obtained highly sparse functional coupling networks with substantially different community structure, more parsimonious encoding models, and decoding models that relied on fewer single-units. CONCLUSIONS: Together, these results demonstrate that improved parameter inference, achieved via UoI, reshapes interpretation in diverse neuroscience contexts.

High density carbon fiber arrays for chronic electrophysiology, fast scan cyclic voltammetry, and correlative anatomy.

OBJECTIVE: Multimodal measurements at the neuronal level allow for detailed insight into local circuit function. However, most behavioral studies focus on one or two modalities and are generally limited by the available technology. APPROACH: Here, we show a combined approach of electrophysiology recordings, chemical sensing, and histological localization of the electrode tips within tissue. The key enabling technology is the underlying use of carbon fiber electrodes, which are small, electrically conductive, and sensitive to dopamine. The carbon fibers were functionalized by coating with Parylene C, a thin insulator with a high dielectric constant, coupled with selective re-exposure of the carbon surface using laser ablation. MAIN RESULTS: We demonstrate the use of this technology by implanting 16 channel arrays in the rat nucleus accumbens. Chronic electrophysiology and dopamine signals were detected 1 month post implant. Additionally, electrodes were left in the tissue, sliced in place during histology, and showed minimal tissue damage. SIGNIFICANCE: Our results validate our new technology and methods, which will enable a more comprehensive circuit level understanding of the brain.

An expanded palette of dopamine sensors for multiplex imaging in vivo.

Genetically encoded dopamine sensors based on green fluorescent protein (GFP) enable high-resolution imaging of dopamine dynamics in behaving animals. However, these GFP-based variants cannot be readily combined with commonly used optical sensors and actuators, due to spectral overlap. We therefore engineered red-shifted variants of dopamine sensors called RdLight1, based on mApple. RdLight1 can be combined with GFP-based sensors with minimal interference and shows high photostability, permitting prolonged continuous imaging. We demonstrate the utility of RdLight1 for receptor-specific pharmacological analysis in cell culture, simultaneous assessment of dopamine release and cell-type-specific neuronal activity and simultaneous subsecond monitoring of multiple neurotransmitters in freely behaving rats. Dual-color photometry revealed that dopamine release in the nucleus accumbens evoked by reward-predictive cues is accompanied by a rapid suppression of glutamate release. By enabling multiplexed imaging of dopamine with other circuit components in vivo, RdLight1 opens avenues for understanding many aspects of dopamine biology.

Cellular-scale silicon probes for high-density, precisely localized neurophysiology.

Neural implants with large numbers of electrodes have become an important tool for examining brain functions. However, these devices typically displace a large intracranial volume compared with the neurons they record. This large size limits the density of implants, provokes tissue reactions that degrade chronic performance, and impedes the ability to accurately visualize recording sites within intact circuits. Here we report next-generation silicon-based neural probes at a cellular scale (5 × 10 µm cross section), with ultra-high-density packing (as little as 66 µm between shanks) and 64 or 256 closely spaced recording sites per probe. We show that these probes can be inserted into superficial or deep brain structures and record large spikes in freely behaving rats for many weeks. Finally, we demonstrate a slice-in-place approach for the precise registration of recording sites relative to nearby neurons and anatomical features, including striatal µ-opioid receptor patches. This scalable technology provides a valuable tool for examining information processing within neural circuits and potentially for human brain-machine interfaces.NEW & NOTEWORTHY Devices with many electrodes penetrating into the brain are an important tool for investigating neural information processing, but they are typically large compared with neurons. This results in substantial damage and makes it harder to reconstruct recording locations within brain circuits. This paper presents high-channel-count silicon probes with much smaller features and a method for slicing through probe, brain, and skull all together. This allows probe tips to be directly observed relative to immunohistochemical markers.

Globus pallidus dynamics reveal covert strategies for behavioral inhibition.

Flexible behavior requires restraint of actions that are no longer appropriate. This behavioral inhibition critically relies on frontal cortex - basal ganglia circuits. Within the basal ganglia, the globus pallidus pars externa (GPe) has been hypothesized to mediate selective proactive inhibition: being prepared to stop a specific action, if needed. Here we investigate population dynamics of rat GPe neurons during preparation-to-stop, stopping, and going. Rats selectively engaged proactive inhibition towards specific actions, as shown by slowed reaction times (RTs). Under proactive inhibition, GPe population activity occupied state-space locations farther from the trajectory followed during normal movement initiation. Furthermore, the state-space locations were predictive of distinct types of errors: failures-to-stop, failures-to-go, and incorrect choices. Slowed RTs on correct proactive trials reflected starting bias towards the alternative action, which was overcome before progressing towards action initiation. Our results demonstrate that rats can exert cognitive control via strategic adjustments to their GPe network state.

The Degree of Nesting between Spindles and Slow Oscillations Modulates Neural Synchrony.

Spindles and slow oscillations (SOs) both appear to play an important role in memory consolidation. Spindle and SO "nesting," or the temporal overlap between the two events, is believed to modulate consolidation. However, the neurophysiological processes modified by nesting remain poorly understood. We thus recorded activity from the primary motor cortex of 4 male sleeping rats to investigate how SO and spindles interact to modulate the correlation structure of neural firing. During spindles, primary motor cortex neurons fired at a preferred phase, with neural pairs demonstrating greater neural synchrony, or correlated firing, during spindle peaks. We found a direct relationship between the temporal proximity between SO and spindles, and changes to the distribution of neural correlations; nesting was associated with narrowing of the distribution, with a reduction in low- and high-correlation pairs. Such narrowing may be consistent with greater exploration of neural states. Interestingly, after animals practiced a novel motor task, pairwise correlations increased during nested spindles, consistent with targeted strengthening of functional interactions. These findings may be key mechanisms through which spindle nesting supports memory consolidation.SIGNIFICANCE STATEMENT Our analysis revealed changes in cortical spiking structure that followed the waxing and waning of spindles; firing rates increased, spikes were more phase-locked to spindle-band local field potential, and synchrony across units peaked during spindles. Moreover, we showed that the degree of nesting between spindles and slow oscillations modified the correlation structure across units by narrowing the distribution of pairwise correlations. Finally, we demonstrated that engaging in a novel motor task increased pairwise correlations during nested spindles. These phenomena suggest key mechanisms through which the interaction of spindles and slow oscillations may support sensorimotor learning. More broadly, this work helps link large-scale measures of population activity to changes in spiking structure, a critical step in understanding neuroplasticity across multiple scales.

Knock-In Rat Lines with Cre Recombinase at the Dopamine D1 and Adenosine 2a Receptor Loci.

Genetically modified mice have become standard tools in neuroscience research. Our understanding of the basal ganglia in particular has been greatly assisted by BAC mutants with selective transgene expression in striatal neurons forming the direct or indirect pathways. However, for more sophisticated behavioral tasks and larger intracranial implants, rat models are preferred. Furthermore, BAC lines can show variable expression patterns depending upon genomic insertion site. We therefore used CRISPR/Cas9 to generate two novel knock-in rat lines specifically encoding Cre recombinase immediately after the dopamine D1 receptor (Drd1a) or adenosine 2a receptor (Adora2a) loci. Here, we validate these lines using in situ hybridization and viral vector mediated transfection to demonstrate selective, functional Cre expression in the striatal direct and indirect pathways, respectively. We used whole-genome sequencing to confirm the lack of off-target effects and established that both rat lines have normal locomotor activity and learning in simple instrumental and Pavlovian tasks. We expect these new D1-Cre and A2a-Cre rat lines will be widely used to study both normal brain functions and neurological and psychiatric pathophysiology.

Publisher Correction: Dissociable dopamine dynamics for learning and motivation.

Change history: In this Article, an extraneous label appeared in Fig. 4b, and has been removed in the online version.

Dissociable dopamine dynamics for learning and motivation.

The dopamine projection from ventral tegmental area (VTA) to nucleus accumbens (NAc) is critical for motivation to work for rewards and reward-driven learning. How dopamine supports both functions is unclear. Dopamine cell spiking can encode prediction errors, which are vital learning signals in computational theories of adaptive behaviour. By contrast, dopamine release ramps up as animals approach rewards, mirroring reward expectation. This mismatch might reflect differences in behavioural tasks, slower changes in dopamine cell spiking or spike-independent modulation of dopamine release. Here we compare spiking of identified VTA dopamine cells with NAc dopamine release in the same decision-making task. Cues that indicate an upcoming reward increased both spiking and release. However, NAc core dopamine release also covaried with dynamically evolving reward expectations, without corresponding changes in VTA dopamine cell spiking. Our results suggest a fundamental difference in how dopamine release is regulated to achieve distinct functions: broadcast burst signals promote learning, whereas local control drives motivation.

What does dopamine mean?

Dopamine is a critical modulator of both learning and motivation. This presents a problem: how can target cells know whether increased dopamine is a signal to learn or to move? It is often presumed that motivation involves slow ('tonic') dopamine changes, while fast ('phasic') dopamine fluctuations convey reward prediction errors for learning. Yet recent studies have shown that dopamine conveys motivational value and promotes movement even on subsecond timescales. Here I describe an alternative account of how dopamine regulates ongoing behavior. Dopamine release related to motivation is rapidly and locally sculpted by receptors on dopamine terminals, independently from dopamine cell firing. Target neurons abruptly switch between learning and performance modes, with striatal cholinergic interneurons providing one candidate switch mechanism. The behavioral impact of dopamine varies by subregion, but in each case dopamine provides a dynamic estimate of whether it is worth expending a limited internal resource, such as energy, attention, or time.